AI assistant

Integrated AI pair programmer powered by Cloudflare Workers AI

Overview

Duet includes an AI assistant that acts as an additional pair programmer in your session. It’s powered by Meta’s Llama 3 8B model running on Cloudflare’s edge network, providing fast responses without leaving your terminal.

Activating the AI

Press Ctrl+G while in a room to open the AI input prompt:

1. Open AI input

   Press Ctrl+G in the terminal. You’ll see:

   Ask the AI... |
   
2. Type your question

   Ask anything about your code, request help with commands, or get debugging suggestions:

   How do I list all files modified in the last hour?
   
3. Submit with Enter

   The AI will respond in the sidebar. Press Esc to cancel without sending.

The AI assistant requires a Cloudflare Worker URL to be configured when starting the Duet server:

duet --worker https://duet-cf-worker.your-subdomain.workers.dev

AI architecture

The AI runs as a Cloudflare Durable Object, maintaining conversation state per room:

export class DuetAgent extends Agent<Env, DuetAgentState> {
  override initialState: DuetAgentState = { messages: [] };

  private async runAI(messages: AIMessage[]): Promise<string> {
    const result = await this.env.AI.run("@cf/meta/llama-3-8b-instruct", {
      messages,
    });
    return result.response?.trim() || "";
  }
}

Model

Llama 3 8B Instruct

• 8 billion parameter model
• Optimized for instruction following
• Runs on Cloudflare’s global network
• Typical response time: 1-3 seconds

Context window

Duet keeps the last 10 messages in context:

...this.state.messages.slice(-10).map<AIMessage>((m) => ({
  role: m.role === "agent" ? "assistant" : "user",
  content: m.text,
}))

This balances memory usage with conversation coherence.

System prompt

{
  role: "system",
  content:
    "You are Duet, a concise pair-programming assistant. " +
    "You can run commands in a sandbox using <run>command</run> tags. " +
    "When asked to perform an action, briefly explain what you will do " +
    "and wrap the exact shell command(s) in <run> tags. " +
    "Do NOT include predicted output in your response - just provide " +
    "the explanation and command."
}

AI sidebar

The AI chat appears in a collapsible sidebar on the right side of your terminal:

┌─────────────────┬──────────────────────┬────────────────────┐
│   Users         │   Terminal           │   AI Assistant     │
│                 │                      │                    │
│   alice (host)  │   $ ls -la           │   You: How do I... │
│   bob           │   total 48           │                    │
│                 │   drwxr-xr-x ...     │   AI: You can use..│
└─────────────────┴──────────────────────┴────────────────────┘

Keyboard shortcuts

Key	Action
Ctrl+A	Toggle AI sidebar visibility
Ctrl+J	Scroll chat down (3 lines)
Ctrl+K	Scroll chat up (3 lines)
Ctrl+G	Open AI input prompt

The AI sidebar only appears when your terminal is at least 120 columns wide and 24 rows tall. On smaller windows, the feature is automatically disabled.

Command execution

The AI can execute commands in a Cloudflare Sandbox by wrapping them in <run> tags:

private async executeCommands(text: string, roomId: string): Promise<string> {
  const matches = Array.from(text.matchAll(/<run>([\s\S]*?)<\/run>/g));
  let result = text;

  for (const match of matches) {
    const cmd = match[1]?.trim();
    if (!cmd) continue;

    try {
      const sandbox = getSandbox(this.env.Sandbox, `sandbox-${roomId}`);
      const { stderr, stdout } = await sandbox.exec(cmd);

      const summary = stdout.slice(0, 500) || stderr.slice(0, 500) || "[no output]";
      result += `\n\nOutput (${cmd}):\n${summary}`;
    } catch (e) {
      result += `\n\nError (${cmd}):\n${e.message}`;
    }
  }
  return result.replace(/<run>[\s\S]*?<\/run>/g, "").trim();
}

Example conversation

User

How do I find all Python files?

Commands run in an isolated Cloudflare Sandbox, not your shared terminal workspace. The sandbox is ephemeral and destroyed when the room ends.

Conversation persistence

AI messages are synchronized across all participants in real time:

case AIResponseMsg:
    if m.currentRoom != nil {
        m.currentRoom.SetAIMessages(msg.Messages)
        // Notify other clients to sync their viewport
        m.currentRoom.BroadcastEvent(room.RoomEvent{
            Type: "ai_sync",
        }, m.clientID)
    }
    m.syncAIViewportContent()
    m.scrollToLastPrompt()

When someone asks the AI a question:

1. The response is stored in the room’s shared state
2. An ai_sync event is broadcast to all participants
3. Everyone’s AI sidebar updates to show the new messages

Late joiners can see the full AI conversation history when they enter the room. The last 50 messages are kept in memory.

Message format

Each message in the conversation includes:

type AIMessage struct {
    Role   string `json:"role"`    // "user" or "agent"
    UserID string `json:"user_id"` // Username of the person who asked
    Text   string `json:"text"`    // Message content
    Ts     int64  `json:"ts"`      // Unix timestamp
}

This allows the UI to display who asked each question:

You: How do I list files?

AI: Use the ls command...

alice: What about sorting by date?

AI: Add the -t flag...

API endpoints

The Cloudflare Worker exposes these AI endpoints:

POST /api/rooms/:roomId/message · endpoint

Send a message to the AI

Request body:

{
  "text": "How do I list files?",
  "userId": "alice"
}

Response:

{
  "reply": "Use the ls command to list files...",
  "messages": [
    {"role": "user", "userId": "alice", "text": "How do I...", "ts": 1234567890},
    {"role": "agent", "text": "Use the ls command...", "ts": 1234567891}
  ]
}

DELETE /api/rooms/:roomId · endpoint

Clean up AI state and sandbox for a room

Called automatically when the last participant leaves.

Response:

{
  "cleaned": true,
  "roomId": "a3f8e9d2-..."
}

Client implementation

The Go client communicates with the Worker:

func (c *Client) SendMessage(ctx context.Context, roomID, text, userID string) (*MessageResponse, error) {
    url := fmt.Sprintf("%s/api/rooms/%s/message", c.baseURL, roomID)

    body := MessageRequest{
        Text:   text,
        UserID: userID,
    }

    jsonBody, err := json.Marshal(body)
    req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(jsonBody))
    req.Header.Set("Content-Type", "application/json")

    resp, err := c.http.Do(req)
    // ... handle response
}

Requests timeout after 30 seconds to prevent hanging if the Worker is slow.

Error handling

No Worker URL configured

If you press Ctrl+G without a Worker URL:

AI not configured (no worker URL)

Start the server with the --worker flag.

Request timeout

If the AI doesn’t respond within 30 seconds:

Error: context deadline exceeded

Try again or check your Worker status.

Validation error

Empty messages are rejected:

{
  "error": "invalid request",
  "details": {"text": ["Text cannot be empty"]}
}

Best practices

Be specific

Instead of “help with this code”, ask “how do I parse JSON in Go?”

One question at a time

The AI works best with focused questions. Break complex tasks into steps.

Include context

Mention what you’re trying to do: “I’m debugging a Python script that…”

Review commands

Always verify AI-suggested commands before running them in your terminal.

Limitations

• The AI doesn’t have access to your terminal history or current directory
• It can’t see files in your workspace (only in its own sandbox)
• It doesn’t remember conversations across different rooms
• The 10-message context window means very long discussions may lose coherence

Next steps

Sandbox execution

Learn how the AI’s command execution sandbox works

Deploy a Worker

Set up your own Cloudflare Worker for AI features